fix(quality-gates): fail coderabbit on non-zero exit by rafaelscosta · Pull Request #767 · SynkraAI/aiox-core

rafaelscosta · 2026-05-18T15:37:59Z

Summary

Fail Layer2 CodeRabbit gate when runCommand returns a non-zero exit code, even if stdout/stderr are empty.
Mirror the non-zero exit handling in workflow-executor CodeRabbit analysis and parse stderr together with stdout.
Add regression coverage for workflow-executor WSL probe behavior and CodeRabbit non-zero exits.

Validation

npm ci --ignore-scripts
npx jest --runInBand --runTestsByPath tests/unit/quality-gates/layer2-pr-automation.test.js tests/unit/quality-gates/cross-platform-coderabbit.test.js tests/core/workflow-executor.test.js tests/core/orchestration/workflow-executor-coderabbit.test.js
npx eslint --no-ignore .aiox-core/core/quality-gates/layer2-pr-automation.js .aiox-core/core/orchestration/workflow-executor.js tests/unit/quality-gates/layer2-pr-automation.test.js tests/core/orchestration/workflow-executor-coderabbit.test.js
npm run typecheck
npm run lint
npm test -- --runInBand

Fixes #763
Fixes #764

Summary by CodeRabbit

Bug Fixes
- Code quality checks now fail when the analyzer exits with a non-zero code and include the exit code plus truncated output for diagnostics.
- Clarified CLI failure messages and WSL troubleshooting guidance for clearer, platform-specific error guidance.
- Issue parsing now considers combined stdout/stderr to improve detection accuracy.
Tests
- Added regression and unit tests covering non-zero exits and platform-specific invocation behaviors.

vercel · 2026-05-18T15:38:05Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
aiox-core	Ready	Preview, Comment	May 18, 2026 3:41pm

coderabbitai · 2026-05-18T15:38:41Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: b235b50d-5317-4ee6-8681-ca69a218a560

📥 Commits

Reviewing files that changed from the base of the PR and between f84375b and b9e242d.

📒 Files selected for processing (5)

.aiox-core/core/orchestration/workflow-executor.js
.aiox-core/core/quality-gates/layer2-pr-automation.js
.aiox-core/install-manifest.yaml
tests/core/orchestration/workflow-executor-coderabbit.test.js
tests/unit/quality-gates/layer2-pr-automation.test.js

✅ Files skipped from review due to trivial changes (1)

.aiox-core/install-manifest.yaml

🚧 Files skipped from review as they are similar to previous changes (3)

tests/unit/quality-gates/layer2-pr-automation.test.js
.aiox-core/core/orchestration/workflow-executor.js
tests/core/orchestration/workflow-executor-coderabbit.test.js

Walkthrough

This PR implements exit code validation for CodeRabbit CLI invocations in both the workflow executor and quality-gates automation layer. It adds comprehensive test coverage for the executor's WSL probe behavior and error paths, plus validation that non-zero exits trigger failures instead of silent pass results.

Changes

Non-zero exit handling and test coverage

Layer / File(s)	Summary
Executor and automation exit code error handling `.aiox-core/core/orchestration/workflow-executor.js`, `.aiox-core/core/quality-gates/layer2-pr-automation.js`	Non-zero CodeRabbit exit codes now surface as errors with captured exit code and truncated stdout/stderr; issue parsing receives combined stdout+stderr instead of stdout alone. WSL error messaging is refined.
Executor CodeRabbit invocation test suite `tests/core/orchestration/workflow-executor-coderabbit.test.js`	New Jest test suite mocks `child_process` operations to verify executor detects non-zero exits with error details, covers WSL probe failure modes (missing ENOENT and no usable distributions), and validates native mode avoids WSL probing on Windows and macOS.
Automation layer exit code test `tests/unit/quality-gates/layer2-pr-automation.test.js`	New test case confirms `runCodeRabbit` fails with informative error when CodeRabbit command returns non-zero exit code with empty output.

Sequence Diagram

sequenceDiagram
  participant Test
  participant Executor as WorkflowExecutor
  participant SpawnSync as spawnSync (WSL probe)
  participant Exec as exec (CodeRabbit)

  Test->>Executor: runCodeRabbitAnalysis with installation_mode='wsl'
  Executor->>SpawnSync: probe WSL availability
  SpawnSync-->>Executor: error ENOENT or status≠0
  Executor-->>Test: return success:false + WSL guidance

  Test->>Executor: runCodeRabbitAnalysis with installation_mode='native'
  Executor->>Exec: run CodeRabbit CLI
  Exec-->>Executor: { exitCode: 137, stdout, stderr }
  Executor-->>Test: return success:false + exit code in error

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

SynkraAI/aiox-core#752: Both PRs change CodeRabbit execution in the same core flows (runCodeRabbitAnalysis in workflow-executor.js and runCodeRabbit in layer2-pr-automation.js).
SynkraAI/aiox-core#761: Prior PR that introduced WSL probe path in both executor and automation layer; this PR adds error handling and test coverage for that probe.

Suggested reviewers

Pedrovaleriolopez
oalanicolas

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'fix(quality-gates): fail coderabbit on non-zero exit' directly and clearly summarizes the main change: enforcing non-zero exit code handling in the quality-gates module.
Linked Issues check	✅ Passed	The pull request comprehensively addresses all coding objectives from both linked issues: non-zero exit code checking in runCodeRabbit [`#763`], mirrored handling in workflow-executor [`#763`], and new WSL probe tests for workflow-executor [`#764`].
Out of Scope Changes check	✅ Passed	All changes are directly aligned with the linked issues. Modifications to workflow-executor.js and layer2-pr-automation.js implement the required exit code checks; new tests validate both non-zero exits and WSL probe behavior; and the manifest update reflects these code changes.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch fix/coderabbit-exitcode-wsl-tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

github-actions · 2026-05-18T15:39:44Z

📊 Coverage Report

Coverage report not available

📈 Full coverage report available in Codecov

Generated by PR Automation (Story 6.1)

coderabbitai

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.aiox-core/core/orchestration/workflow-executor.js:
- Line 832: The concatenation of stdout and stderr passed to
parseCodeRabbitOutput can merge the last line of stdout with the first line of
stderr; change the call in workflow-executor.js that builds the input for
parseCodeRabbitOutput (the expression currently using `${stdout || ''}${stderr
|| ''}`) to insert a newline separator (e.g., `${stdout || ''}\n${stderr ||
''}`) so stream boundaries are preserved before calling
this.parseCodeRabbitOutput.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 9835c869-0e91-4692-aa53-d8646c96ede6

📥 Commits

Reviewing files that changed from the base of the PR and between f31254d and f84375b.

📒 Files selected for processing (5)

.aiox-core/core/orchestration/workflow-executor.js
.aiox-core/core/quality-gates/layer2-pr-automation.js
.aiox-core/install-manifest.yaml
tests/core/orchestration/workflow-executor-coderabbit.test.js
tests/unit/quality-gates/layer2-pr-automation.test.js

rafaelscosta requested review from Pedrovaleriolopez and oalanicolas as code owners May 18, 2026 15:38

github-actions Bot added type: test Test coverage and quality area: core Core framework (.aios-core/core/) area: docs Documentation (docs/) labels May 18, 2026

coderabbitai Bot requested changes May 18, 2026

View reviewed changes

Comment thread .aiox-core/core/orchestration/workflow-executor.js Outdated

rafaelscosta force-pushed the fix/coderabbit-exitcode-wsl-tests branch from f84375b to 15269ec Compare May 18, 2026 15:41

vercel Bot deployed to Preview May 18, 2026 15:41 View deployment

fix(quality-gates): fail coderabbit on non-zero exit [#763 #764]

b9e242d

rafaelscosta force-pushed the fix/coderabbit-exitcode-wsl-tests branch from 15269ec to b9e242d Compare May 18, 2026 15:42

coderabbitai Bot approved these changes May 18, 2026

View reviewed changes

rafaelscosta merged commit dc4149d into main May 18, 2026
41 checks passed

rafaelscosta deleted the fix/coderabbit-exitcode-wsl-tests branch May 18, 2026 15:47

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix(quality-gates): fail coderabbit on non-zero exit - #767

fix(quality-gates): fail coderabbit on non-zero exit#767
rafaelscosta merged 1 commit into
mainfrom
fix/coderabbit-exitcode-wsl-tests

rafaelscosta commented May 18, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

vercel Bot commented May 18, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented May 18, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented May 18, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Uh oh!

Conversation

rafaelscosta commented May 18, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Validation

Summary by CodeRabbit

Uh oh!

vercel Bot commented May 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coderabbitai Bot commented May 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Sequence Diagram

Estimated code review effort

Possibly related PRs

Suggested reviewers

Uh oh!

github-actions Bot commented May 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📊 Coverage Report

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

rafaelscosta commented May 18, 2026 •

edited by coderabbitai Bot

Loading

vercel Bot commented May 18, 2026 •

edited

Loading

coderabbitai Bot commented May 18, 2026 •

edited

Loading

github-actions Bot commented May 18, 2026 •

edited

Loading